Eye scanner for user identification and security in an eyewear device

ABSTRACT

A system comprises an eyewear device that includes a frame, a temple connected to a lateral side of the frame, an infrared emitter, and an infrared camera. The infrared emitter and the infrared camera are connected to the frame or the temple to emit a pattern of infrared light. The system includes a processor coupled to the eyewear device, a memory accessible to the processor, and programming in the memory. Execution of the programming by the processor configures the system to perform functions, including functions to emit, via the infrared emitter, a pattern of infrared light on an eye of a user of the eyewear device; capture, via the camera, reflection variations in the pattern of infrared light on the eye of the user; and identify a user of the eyewear device based on the reflection variations of the emitted pattern of infrared light on the eye of the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.16/188,981 filed on Nov. 13, 2018, which claims priority to U.S.Provisional Application Ser. No. 62/588,700 filed on Nov. 20, 2017, thecontents of which are incorporated fully herein by reference.

TECHNICAL FIELD

The present subject matter relates to eye scanners for an eyeweardevice, e.g., smart glasses, for user identification and security.

BACKGROUND

Portable eyewear devices, such as smartglasses, headwear, and headgearavailable today integrate cameras and displays. Users of such portableeyewear devices may share such eyewear devices with friends and familymembers so that any user can borrow the eyewear device to capture imageswith the integrated camera.

Verifying the identity of the specific user of the portable eyeweardevice can be useful. For example, as augmented reality becomes moreprevalent in such eyewear devices applications may be developed thatneed to verify the identity of the user for security purposes.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations, by way ofexample only, not by way of limitations. In the figures, like referencenumerals refer to the same or similar elements.

FIG. 1 is a rear view of an example hardware configuration of an eyeweardevice, which includes an eye scanner on a frame, for use in a systemfor identifying a user of the eyewear device.

FIG. 2 is a rear view of an example hardware configuration of anothereyewear device, which includes an eye scanner on a chunk, for use in asystem for identifying a user of the eyewear device.

FIG. 3 shows a rear perspective sectional view of the eyewear device ofFIG. 1 depicting an infrared camera, a frame front, a frame back, and acircuit board.

FIG. 4 is a cross-sectional view taken through the infrared camera andthe frame of the eyewear device of FIG. 3.

FIG. 5 shows a rear perspective view of the eyewear device of FIG. 1depicting an infrared emitter, an infrared camera, a frame front, aframe back, and a circuit board.

FIG. 6 is a cross-sectional view taken through the infrared emitter andthe frame of the eyewear device of FIG. 5.

FIG. 7 is a top cross-sectional view of the chunk of the eyewear deviceof FIG. 2 depicting the infrared emitter, the infrared camera, and acircuit board.

FIG. 8A depicts an example of a pattern of infrared light emitted by aninfrared emitter of the eyewear device and reflection variations of theemitted pattern of infrared light captured by the infrared camera of theeyewear device.

FIG. 8B depicts the emitted pattern of infrared light being emitted bythe infrared emitter of the eyewear device in an inwards facing field ofview towards an eye of a user.

FIG. 9 is a high-level functional block diagram of an example useridentification system including the eyewear device, a mobile device, anda server system connected via various networks.

FIG. 10 shows an example of a hardware configuration for the mobiledevice of the user identification system of FIG. 9, in simplified blockdiagram form.

FIG. 11A shows various alternate locations for the eye scanner on theeyewear device, which can be used individually or in combination.

FIGS. 11B, 11C, and 11D illustrate the effects of the various alternatelocations on the eyewear device with respect to different orientationsof the eye of the user.

FIG. 12 is a flowchart of the operation of the eyewear device and othercomponents of the user identification system.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, it should be apparent to those skilledin the art that the present teachings may be practiced without suchdetails. In other instances, well known methods, procedures, components,and circuitry have been described at a relatively high-level, withoutdetail, in order to avoid unnecessarily obscuring aspects of the presentteachings.

The term “coupled” as used herein refers to any logical, optical,physical or electrical connection, link or the like by which signals orlight produced or supplied by one system element are imparted to anothercoupled element. Unless described otherwise, coupled elements or devicesare not necessarily directly connected to one another and may beseparated by intermediate components, elements or communication mediathat may modify, manipulate or carry the light or signals.

The orientations of the eyewear device, associated components and anycomplete devices incorporating an eye scanner such as shown in any ofthe drawings, are given by way of example only, for illustration anddiscussion purposes. In operation for a particular variable opticalprocessing application, the eyewear device may be oriented in any otherdirection suitable to the particular application of the eyewear device,for example up, down, sideways, or any other orientation. Also, to theextent used herein, any directional term, such as front, rear, inwards,outwards, towards, left, right, lateral, longitudinal, up, down, upper,lower, top, bottom and side, are used by way of example only, and arenot limiting as to direction or orientation of any optic or component ofan optic constructed as otherwise described herein.

In an example, a system includes an eyewear device. The eyewear deviceincludes a frame and a temple connected to a lateral side of the frame.The eyewear device further includes an infrared emitter connected to theframe or the temple to emit a pattern of infrared light. The eyeweardevice further includes an infrared camera connected to the frame or thetemple to emit a pattern of infrared light. The system further includesa processor coupled to the eyewear device, a memory accessible to theprocessor, and programming in the memory.

Execution of the programming by the processor configures the system toperform functions, including functions to emit, via the infraredemitter, the pattern of infrared light on an eye of a user of theeyewear device. The execution of the programming by the processorfurther configures the system to capture, via the infrared camera, thereflection variations in the emitted pattern of infrared light on theeye of the user. The execution of the programming by the processorfurther configures the system to identify a user, or an account, of theeyewear device based on the reflection variations of the emitted patternof infrared light on the eye of the user.

Additional objects, advantages and novel features of the examples willbe set forth in part in the description which follows, and in part willbecome apparent to those skilled in the art upon examination of thefollowing and the accompanying drawings or may be learned by productionor operation of the examples. The objects and advantages of the presentsubject matter may be realized and attained by means of themethodologies, instrumentalities and combinations particularly pointedout in the appended claims.

Reference now is made in detail to the examples illustrated in theaccompanying drawings and discussed below.

FIG. 1 is a rear view of an example hardware configuration of an eyeweardevice 100, which includes an eye scanner 113 on a frame 105, for use ina system for identifying a user of the eyewear device 100. As shown inFIG. 1, the eyewear device 100 is in a form configured for wearing by auser, which are eyeglasses in the example of FIG. 1. The eyewear device100 can take other forms and may incorporate other types of frameworks,for example, a headgear, a headset, or a helmet.

In the eyeglasses example, eyewear device 100 includes a frame 105 whichincludes a left rim 107A connected to a right rim 107B via a bridge 106adapted for a nose of the user. The left and right rims 107A-B includerespective apertures 175A-B which hold a respective optical element180A-B, such as a lens and a display device. As used herein, the termlens is meant to cover transparent or translucent pieces of glass orplastic having curved and flat surfaces that cause light toconverge/diverge or that cause little or no convergence/divergence.

Although shown as having two optical elements 180A-B, the eyewear device100 can include other arrangements, such as a single optical element ormay not include any optical element 180A-B depending on the applicationor intended user of the eyewear device 100. As further shown, eyeweardevice 100 includes a left chunk 110A adjacent the left lateral side170A of the frame 105 and a right chunk 110B adjacent the right lateralside 170B of the frame 105. The chunks 110A-B may be integrated into theframe 105 on the respective sides 170A-B (as illustrated) or implementedas separate components attached to the frame 105 on the respective sides170A-B. Alternatively, the chunks 110A-B may be integrated into temples(not shown) attached to the frame 105.

In the example of FIG. 1, the eye scanner 113 includes an infraredemitter 115 and an infrared camera 120. Visible light cameras typicallyinclude a blue light filter to block infrared light detection, in anexample, the infrared camera 120 is a visible light camera, such as alow resolution video graphic array (VGA) camera (e.g., 640×480 pixelsfor a total of 0.3 megapixels), with the blue filter removed. Theinfrared emitter 115 and the infrared camera 120 are co-located on theframe 105, for example, both are shown as connected to the upper portionof the left rim 107A. As described in further detail below, the frame105 or one or more of the left and right chunks 110A-B include a circuitboard that includes the infrared emitter 115 and the infrared camera120. The infrared emitter 115 and the infrared camera 120 can beconnected to the circuit board by soldering, for example.

Other arrangements of the infrared emitter 115 and infrared camera 120can be implemented, including arrangements in which the infrared emitter115 and infrared camera 120 are both on the right rim 107B, or indifferent locations on the frame 105, for example, the infrared emitter115 is on the left rim 107A and the infrared camera 120 is on the rightrim 107B. In another example, the infrared emitter 115 is on the frame105 and the infrared camera 120 is on one of the chunks 110A-B, or viceversa. The infrared emitter 115 can be connected essentially anywhere onthe frame 105, left chunk 110A, or right chunk 110B to emit a pattern ofinfrared light. Similarly, the infrared camera 120 can be connectedessentially anywhere on the frame 105, left chunk 110A, or right chunk110B to capture at least one reflection variation in the emitted patternof infrared light.

The infrared emitter 115 and infrared camera 120 are arranged to faceinwards towards the eye of the user with a partial or full field of viewof the eye in order to pick up an infrared image of the eye for identifyverification. For example, the infrared emitter 115 and infrared camera120 are positioned directly in front of the eye, in the upper part ofthe frame 105 or in the chunks 110A-B at either ends of the frame 105.

In an embodiment, the identification establishes globally unique andunambiguous identifiers of the eye, which serves to distinguish adiscrete individual from other like and unlike users. In the example,the eye scanner 113 is a retina scanner which uses infrared lightillumination (e.g., near-infrared, short-wavelength infrared,mid-wavelength infrared, long-wavelength infrared, or far infrared) toidentify the unique blood vessel configuration in the eye, for example,an unperceived beam of low-energy infrared light is cast on the person'seye. Since the pattern of infrared light emitted by the infrared emitter115 is more easily absorbed by the blood vessels than the surroundingtissue, the amount of light reflected back to the infrared camera 120varies, which can then be used to uniquely identify the user. Suchretinal scanning is an ocular-based biometric technology that uses theunique patterns on a person's retina blood vessels. A useridentification algorithm using digital templates encoded from thesepatterns by mathematical and statistical algorithms allow for patternrecognition of the retina blood vessels or iris and hence theidentification of the user. Although identification can be unique, inother embodiments, the identification establishes that the user is partof a group of users. In response to being identified as part of a group,the user can be provided permissions to access, control, or utilize, onemore executable software applications or hardware features (e.g., avisible light camera) of the eyewear device 100.

Although not shown in FIG. 1, the eyewear device 100 is coupled to aprocessor and a memory, for example in the eyewear device 100 itself oranother part of the system. Subsequent processing by the eyewear device100 by the system, for example, using a coupled memory and processor inthe system to process the captured image of reflection variations ofinfrared light from the retina, identifies the unique pattern of theuser's eye and thus the particular user of the eyewear device 100.

Alternatively, or additionally, the eye scanner 113 may include anemitter that emits other wavelengths of light besides infrared and theeye scanner 113 further includes a camera sensitive to that wavelengththat receives and captures images with that wavelength. For example, theeye scanner 113 may comprise a visible light camera that captures lightin the visible light range from the iris. In some examples, such irisrecognition can use infrared illumination by the infrared emitter 115and the infrared camera 120 or a video camera to capture images of thedetail-rich, intricate structures of the iris which are visibleexternally. The eyewear device 100 or the system can subsequentlyprocess images captured of the iris using, for example, a coupled memoryand processor in the system to process the captured image of visiblelight from the iris. Such processing of the captured images can identifythe unique pattern of the user's eye and thus the particular user of theeyewear device 100.

FIG. 2 is a rear view of an example hardware configuration of anothereyewear device 200. In this example configuration, the eyewear device200 is depicted as including an eye scanner 213 on a right chunk 210B.As shown, the infrared emitter 215 and the infrared camera 220 areco-located on the right chunk 210B. It should be understood that the eyescanner 213 or one or more components of the eye scanner 213 can belocated on the left chunk 210A and other locations of the eyewear device200, for example, the frame 205. Eye scanner 213 has an infrared emitter215 and infrared camera 220 like that of FIG. 1, but the eye scanner 213can be varied to be sensitive to different light wavelengths asdescribed previously in FIG. 1.

Similar to FIG. 1, the eyewear device 200 includes a frame 105 whichincludes a left rim 107A which is connected to a right rim 107B via abridge 106; and the left and right rims 107A-B include respectiveapertures which hold a respective optical element 180A-B.

FIG. 3 shows a rear perspective view of the eyewear device of FIG. 1depicting an infrared camera 120, a frame front 330, a frame back 335,and a circuit board. It can be seen that the upper portion of the leftrim 107A of the frame 105 of the eyewear device 100 includes a framefront 330 and a frame back 335. The frame front 330 includes afront-facing side configured to face outwards away from the eye of theuser. The frame back 335 includes a rear-facing side configured to faceinwards towards the eye of the user. An opening for the infrared camera120 is formed on the frame back 335.

As shown in the encircled cross-section 4-4 of the upper middle portionof the left rim of the frame, a circuit board, which is a flexibleprinted circuit board (PCB) 340, is sandwiched between the frame front330 and the frame back 335. Also shown in further detail is theattachment of the left chunk 110A to the left temple 325A via a lefthinge 326A. In some examples, components of the eye scanner, includingthe infrared camera 120, the flexible PCB 340, or other electricalconnectors or contacts may be located on the left temple 325A or theleft hinge 326A.

In an example, the left chunk 110A includes a chunk body 311, a chunkcap 312, an inwards facing surface 391 and an outwards facing surface392 (labeled, but not visible). Disposed inside the left chunk 110A arevarious interconnected circuit boards, such as PCBs or flexible PCBs,that include controller circuits for charging, a battery, inwards facinglight emitting diodes (LEDs), and outwards (forward) facing LEDs.

FIG. 4 is a cross-sectional view through the infrared camera 120 and theframe corresponding to the encircled cross-section 4-4 of the eyeweardevice of FIG. 3. Various layers of the eyewear device 100 are visiblein the cross-section of FIG. 4. As shown the flexible PCB 340 isdisposed on the frame front 330 and connected to the frame back 335. Theinfrared camera 320 is disposed on the flexible PCB 340 and covered byan infrared camera cover lens 445. For example, the infrared camera 120is reflowed to the back of the flexible PCB 340. Reflowing attaches theinfrared camera 120 to electrical contact pad(s) formed on the back ofthe flexible PCB 340 by subjecting the flexible PCB 340 to controlledheat which melts a solder paste to connect the two components. In oneexample, reflowing is used to surface mount the infrared camera 120 onthe flexible PCB 340 and electrically connect the two components.However, it should be understood that through-holes can be used toconnect leads from the infrared camera 120 to the flexible PCB 340 viainterconnects, for example.

The frame back 335 includes an infrared camera opening 450 for theinfrared camera cover lens 445. The infrared camera opening 450 isformed on a rear-facing side of the frame back 335 that is configured toface inwards towards the eye of the user. In the example, the flexiblePCB 340 can be connected to the frame front 330 via a flexible PCBadhesive 460. The infrared camera cover lens 445 can be connected to theframe back 335 via infrared camera cover lens adhesive 455. Theconnection can be indirect via intervening components.

FIG. 5 shows a rear perspective view of the eyewear device of FIG. 1.The eyewear device 100 includes an infrared emitter 115, infrared camera120, a frame front 330, a frame back 335, and a circuit board 340. As inFIG. 3, it can be seen in FIG. 5 that the upper portion of the left rimof the frame of the eyewear device 100 includes the frame front 330 andthe frame back 335. An opening for the infrared emitter 115 is formed onthe frame back 335.

As shown in the encircled cross-section 6-6 in the upper middle portionof the left rim of the frame, a circuit board, which is a flexible PCB340, is sandwiched between the frame front 330 and the frame back 335.Also shown in further detail is the attachment of the left chunk 110A tothe left temple 325A via the left hinge 326A. In some examples,components of the eye scanner, including the infrared emitter 115, theflexible PCB 340, or other electrical connectors or contacts may belocated on the left temple 325A or the left hinge 326A.

FIG. 6 is a cross-sectional view through the infrared emitter 115 andthe frame corresponding to the encircled cross-section 6-6 of theeyewear device of FIG. 5. Multiple layers of the eyewear device 100 areillustrated in the cross-section of FIG. 6, as shown the frame includesthe frame front 330 and the frame back 335. The flexible PCB 340 isdisposed on the frame front 330 and connected to the frame back 335. Theinfrared emitter 115 is disposed on the flexible PCB 340 and covered byan infrared emitter cover lens 645. For example, the infrared emitter115 is reflowed to the back of the flexible PCB 340. Reflowing attachesthe infrared emitter 115 to contact pad(s) formed on the back of theflexible PCB 340 by subjecting the flexible PCB 340 to controlled heatwhich melts a solder paste to connect the two components. In oneexample, reflowing is used to surface mount the infrared emitter 115 onthe flexible PCB 340 and electrically connect the two components.However, it should be understood that through-holes can be used toconnect leads from the infrared emitter 115 to the flexible PCB 340 viainterconnects, for example.

The frame back 335 includes an infrared emitter opening 650 for theinfrared emitter cover lens 645. The infrared emitter opening 650 isformed on a rear-facing side of the frame back 335 that is configured toface inwards towards the eye of the user. In the example, the flexiblePCB 340 can be connected to the frame front 330 via the flexible PCBadhesive 460. The infrared emitter cover lens 645 can be connected tothe frame back 335 via infrared emitter cover lens adhesive 655. Thecoupling can also be indirect via intervening components.

FIG. 7 is a top cross-sectional view of the right chunk 210B of theeyewear device of FIG. 2. As shown, the eyewear device 200 includes theinfrared emitter 215, the infrared camera 220, and a circuit board,which may be a flexible PCB 740. The right chunk 210B is connected to aright temple 725B of the eyewear device 200 via the right hinge 726B. Insome examples, components of the eye scanner, including the infraredemitter 215 and the infrared camera 220, the flexible PCB 740, or otherelectrical connectors or contacts may be located on the right temple725B or the right hinge 726B.

The right chunk 710B includes chunk body 711, an inwards facing surface791, and an outwards facing surface 792. The right chunk 710B alsoincludes a chunk cap (not shown) like the chunk cap 312 for the leftchunk in FIG. 3, but the chunk cap is removed in the cross-section ofFIG. 7. Disposed inside the right chunk 210B are various interconnectedcircuit boards, such as PCBs or flexible PCBs, that include controllercircuits for a visible light camera 714, microphone(s), low-powerwireless circuitry (e.g., for wireless short range network communicationvia Bluetooth™), high-speed wireless circuitry (e.g., for wireless localarea network communication via WiFi).

The visible light camera 714 is disposed on a circuit board and coveredby a visible camera cover lens and has an outwards facing field of view.The frame front, which is connected to the right chunk 210B, and theright chunk 210B can include opening(s) for the visible light cameracover lens. The frame front includes a front-facing side configured toface outwards away from the eye of the user. The opening for the visiblelight camera cover lens is formed on and through the front-facing side.The infrared emitter 215 and infrared camera 220 have an inwards facingfield of view relative to the visible light camera 714 having theoutwards facing field of view.

As shown, the infrared emitter 215 and the infrared camera 220 areco-located on the inwards facing surface 791 of the right chunk 210B topoint inwards towards the eye of the user. The inwards facing surface791 can be sloped such that it curves away from the upper portion of theright rim of the frame where the inwards facing surface 791 intersectsthe right rim and towards the right temple 725B to orient the infraredemitter 215 and infrared camera 220 with an inwards facing field of viewand a line of sight of the eye of the user.

The infrared emitter 215 and the infrared camera 220 are coupled to theflexible PCB 740 in a manner that is similar to that shown and describedwith reference to FIGS. 3-6. For example, the flexible PCB 740 isdisposed inside the right chunk 210B between inwards facing surface 791and the outwards facing surface 792 of the right chunk 210B. FlexiblePCB 740 is coupled to one or more other components housed in the rightchunk 210B. The infrared emitter 215 is disposed on the flexible PCB 740and an infrared emitter cover lens covers the infrared emitter 215. Theinfrared camera 220 is also disposed on the flexible PCB 740 and aninfrared camera cover lens covers the infrared emitter 215. Althoughshown as being formed on the circuit boards of the right chunk 210B, theeye scanner, including the infrared emitter 215 and the infrared camera220, can be formed on the circuit boards of the left chunk as shown inFIG. 3.

An infrared camera opening and infrared emitter opening are both formedon the inwards facing surface 791 of the right chunk 210B that areconfigured to face inwards towards the eye of the user. In the example,the flexible PCB 740 can be connected to the inwards facing surface 791and outwards facing surface 792 via a flexible PCB adhesive. Theinfrared emitter cover lens and infrared camera cover lens can beconnected to the inwards facing surface 791 via a cover lens adhesive.The coupling can also be indirect via intervening components.

FIG. 8A depicts an example of a pattern of infrared light emitted by aninfrared emitter 815 of the eyewear device and reflection variations ofthe emitted pattern of infrared light captured by the infrared camera820 of the eyewear device. FIG. 8B depicts the emitted pattern ofinfrared light 881 emitted by the infrared emitter 815 of the eyeweardevice in an inwards facing field of view towards an eye of a user 880.

The pattern of infrared light 881 can be a standardized matrix or beamof pixels that will outline a uniform light trace on the eye of the user880 (e.g., retina or iris). As noted above, the eye of each user 880 isunique, for example both the retina and iris portions uniquely identifya user. The retina is a thin tissue composed of neural cells located inthe posterior portion of the eye. Capillaries that supply the retinawith blood form a complex structure that make each user's retina uniqueto that person. The intricate structures forming the iris are alsounique to each person and thus also uniquely identify each user. Whenthe emitted pattern of infrared light 881 strikes the eye of the user880, the infrared camera 820 captures the reflection variations of theemitted pattern of infrared light 882, which can then be used touniquely identify the user.

In an example, the emitted pattern of infrared light 881 is anunperceived low-energy infrared beam that shines on the eye with astandardized path. The amount of reflection of the emitted pattern ofinfrared light 881 varies in different parts of the retina (e.g.,retinal blood vessels absorb light more than surrounding tissue) and theiris. Infrared camera 820 captures these reflection variations of theemitted pattern of infrared light 882, which is digitized by thecomponents of the system. For example, the wearable device includes oris coupled to image processor, memory, and processor for digitizing thereflection variations of the emitted pattern of infrared light 882. Thereflection variations of the emitted pattern of infrared light 882 canthen be compared to a database of captured infrared images of eyes ofmultiple users to identify the user.

To initially set up the user in the system, the reflection variations ofthe emitted pattern of infrared light 882 from the user's eye can bestored in the database of captured infrared images, which includesimages of eyes of multiple users. The system may then subsequentlycompare received reflection variations to this database to uniquelyidentify the user. In an example, when the user is utilizing an eyeweardevice for the first time, the infrared emitter 815 emits the emittedpattern of infrared light 881 and the infrared camera 820 captures one,two, three, or more images of the reflection variations of the emittedpattern of infrared light 882 in different parts of the user's eye(s).If this is the first time the user has used the system, the system willfind no previously captured infrared image exists in the database thatmatches the currently captured reflection variations of the emittedpattern of infrared light 882. In response to finding no matchingcaptured infrared image exists, the system updates the database to storedigitized images of the currently captured reflection variations of theemitted pattern of infrared light 882. During a subsequent use of theeyewear device at a later time, the updated database with the digitizedreflection variations that were previously stored in the database areanalyzed using algorithms. The algorithms employ mathematical andstatistical techniques for pattern recognition to determine whether atleast one subsequently captured image of reflection variations of thatsame user or a different user of the eyewear device matches one or moreof the previously captured digitized images that are stored and exist inthe database. If a match is found, the identity of the user is verified(e.g., known) and corresponding user account information is retrieved. Achat application stored on a mobile device may be executed by aprocessor of the mobile device and utilize the corresponding useraccount information to post or send images and videos captured by avisible light to camera of the eyewear device to the user's account anddeliver the images and videos captured by the visible light camera tocontacts or associated groups of the verified user in the chatapplication. Although the above example describes verifying the identityof the user as knowing their identity or identifying an associated useraccount, some embodiments can include determining that the same personhas used the eyewear device before without specifically knowing theidentify or account information of the user. It should be understoodthat the foregoing functionality can be embodied in programminginstructions of a user identification application found in one or morecomponents of the system.

FIG. 9 is a high-level functional block diagram of an example useridentification system. The system 900 includes eyewear device 910,mobile device 990, and server system 998. Mobile device 990 may be asmartphone, tablet, laptop computer, access point, or any other suchdevice capable of connecting with eyewear device 910 using both alow-power wireless connection 925 and a high-speed wireless connection937. Mobile device 990 is connected to server system 998 and network995. The network 995 may include any combination of wired and wirelessconnections.

Server system 998 may be one or more computing devices as part of aservice or network computing system, for example, that include aprocessor, a memory, and network communication interface to communicateover the network 995 with the mobile device 990 and eyewear device 910.The memory of the server system 998 can include digital images of thereflection variations of the emitted pattern of infrared light ascaptured by the eyewear device 910 and transmitted via the depictednetworks 925, 937, 995. The memory of the server system 998 can alsoinclude a database of captured infrared images of eyes of multiple usersand a user identification application to perform functions of theprogramming described herein. Execution of the programming by theprocessor of the server system 998 can cause the server system 998 toperform some or all of the functions described herein, for example, touniquely identify the user of the eyewear device 910 based on thereflection variations.

Mobile device 990 and elements of network 995, low-power wirelessconnection 925, and high-speed wireless architecture 937 may beimplemented using details of the architecture of mobile device 990, forexample utilizing the short range XCVRs and WWAN XCVRs of mobile device990 described in FIG. 10.

System 900 may optionally include additional peripheral device elements919 and a display 911 integrated with eyewear device 910. Suchperipheral device elements 919 may include biometric sensors, additionalsensors, or display elements integrated with eyewear device 910. Forexample, peripheral device elements 919 may include any I/O componentsincluding output components, motion components, position components, orany other such elements described herein.

Output components include visual components (e.g., a display such as aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), or a projector), acoustic components(e.g., speakers), haptic components (e.g., a vibratory motor), othersignal generators, and so forth. The input components includealphanumeric input components (e.g., a keyboard, a touch screenconfigured to receive alphanumeric input, a photo-optical keyboard, orother alphanumeric input components), point-based input components(e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, orother pointing instruments), tactile input components (e.g., a physicalbutton, a touch screen that provides location and force of touches ortouch gestures, or other tactile input components), audio inputcomponents (e.g., a microphone), and the like.

For example, the biometric components include components to detectexpressions (e.g., hand expressions, facial expressions, vocalexpressions, body gestures, or eye tracking), measure biosignals (e.g.,blood pressure, heart rate, body temperature, perspiration, or brainwaves), identify a person (e.g., voice identification, retinalidentification, facial identification, fingerprint identification, orelectroencephalogram based identification), and the like. The motioncomponents include acceleration sensor components (e.g., accelerometer),gravitation sensor components, rotation sensor components (e.g.,gyroscope), and so forth. The position components include locationsensor components to generate location coordinates (e.g., a GlobalPositioning System (GPS) receiver component), WiFi or Bluetooth™transceivers to generate positioning system coordinates, altitude sensorcomponents (e.g., altimeters or barometers that detect air pressure fromwhich altitude may be derived), orientation sensor components (e.g.,magnetometers), and the like. Such location coordinates can also bereceived over wireless connections 925 and 937 from the mobile device990 via the low-power wireless circuitry 924 or high-speed wirelesscircuitry 936.

Eyewear device 910 includes a visible light camera 914, infrared emitter915, infrared camera 920, image processor 912, interface 916, low-powercircuitry 920, and high-speed circuitry 930. The components shown inFIG. 9 for the eyewear device 910 are located on one or more circuitboards, for example a PCB or flexible PCB, in the chunks or frames.Alternatively or additionally, the depicted components can be located inthe temples, hinges, or bridge of the eyewear device 910.

The infrared camera 920 may be a low resolution camera, such as VGA(640×480 resolution), which can provide for low power consumption sincefewer pixels equals less power and also allows the camera module packageto be small enough to fit into the design of the eyewear device 910,including the frame and chunks. Infrared camera 920 and visible lightcamera 914 can include digital camera elements such as a charge coupleddevice, a lens, or any other respective visible or infrared lightcapturing elements that may be used to capture data.

Interface 916 refers to any source of a user command that is provided toeyewear device 910. In one implementation, interface 916 is a respectivephysical button on a visible light camera 914, infrared emitter 915, orinfrared camera 920 that, when depressed, sends a user input signal frominterface 916 to low power processor 922. In some examples, theinterface 916 is located on different portions of the eyewear device910, such as on a different chunk or the frame, but is electricallyconnected via a circuit board to the visible light camera 914, infraredemitter 915, or infrared camera 920. Interaction with the interface bythe user, e.g., tactile input or a depression of a button followed by animmediate release, can be processed by low power processor 922 as arequest to capture a single image. A depression of such a camera buttonfor a first period of time may be processed by low-power processor 922as a request to capture video data while the button is depressed, and tocease video capture when the button is released, with the video capturedwhile the button was depressed stored as a single video file. In certainembodiments, the low-power processor 922 may have a threshold timeperiod between the press of a button and a release, such as 500milliseconds or one second, below which the button press and release isprocessed as an image request, and above which the button press andrelease is interpreted as a video request.

Use of the interface 916 on the eyewear device 910 can be immediatelyfollowed by user identification via an eye scanner. For example, theinfrared emitter 915 emits a pattern of infrared light and the infraredcamera 920 captures reflection variations in the emitted pattern ofinfrared light by capturing various infrared images. Such useridentification can occur prior to each image request or video requestvia the interface 916 or after a predetermined time interval of usage ofthe eyewear device 910 has elapsed since the user was previouslyidentified via the eye scanner. The low power processor 922 may makethis user identification determination while the video or imageprocessor 912 is booting. In other embodiments, the interface 916 may bea touch screen device, capacitive or resistive strip or array on acircuit board, or any mechanical switch or physical interface capable ofaccepting user inputs associated with a request for data from thevisible light camera 914, infrared emitter 915, or infrared camera 920.In other embodiments, the interface 916 may have a software component,or may be associated with a command received wirelessly from anothersource.

Image processor 912 includes circuitry to receive signals from thevisible light camera 914 and infrared camera 920 and process thosesignals from the visible light camera 914 and infrared camera 920 into aformat suitable for storage in the memory 934. The memory 934 includesvarious images containing reflection variations 960 of the emittedpattern of infrared light of the eye of the user as captured by theinfrared camera 920. In some examples, the memory 934 can also include adatabase of captured infrared images 950 of eyes of multiple users and auser identification application 945 to perform the functions of theprogramming described herein, for example the operations outlined infurther detail in FIGS. 1-8 and 10-12, for example.

As explained in further detail herein, uniquely identifying the userincludes comparing the images containing the reflection variations ofthe emitted pattern of infrared light 960 of the eye of the user againstthe database of captured infrared images of eyes of multiple users 950via a user identification application 945. Such comparison can be doneon a device separate from the eyewear device 910, such as a hostcomputer, which includes the mobile device 990 and server system 998.Due to the private nature of data from retina and iris scans, in someexamples, identification may occur on the eyewear device 910 alone andin combination with the mobile device 990. However, it should beunderstood that user identification can occur on essentially any hostcomputer, which includes both the mobile device 990 and server system998. For example, as shown, the eyewear device 910 can include theprocessors 922, 932; the memory 934, a user identification application945 in the memory 934, to perform the functions of the programming toemit and capture as described herein. The host computer 990 and 998coupled to the eyewear device 910 via the networks 925, 937, and 995 asshown, can include a second processor, a second memory; and the functionof the programming to uniquely identify the user of the eyewear device.Where and which components of the depicted system 900 perform the useridentification, depends on the security preferences of the user andprivacy requirements of the system 900 because storage of such privateidentification data may be subject to various rules and regulations.

Image processor 912 is structured within eyewear device 910 such that itmay be powered on and booted under the control of low-power circuitry920. Image processor 912 may additionally be powered down by low-powercircuitry 920. Depending on various power design elements associatedwith image processor 912, image processor 912 may still consume a smallamount of power even when it is in an off state. This power will,however, be negligible compared to the power used by image processor 912when it is in an on state, and will also have a negligible impact onbattery life. As described herein, device elements in an “off” state arestill configured within a device such that low-power processor 922 isable to power on and power down the devices. A device that is referredto as “off” or “powered down” during operation of eyewear device 910does not necessarily consume zero power due to leakage or other aspectsof a system design.

In one example embodiment, image processor 912 comprises amicroprocessor integrated circuit (IC) customized for processing sensordata from a visible light camera 914 and an infrared camera 920, alongwith volatile memory used by the microprocessor to operate. In order toreduce the amount of time that image processor 912 takes when poweringon to processing data, a non-volatile read only memory (ROM) may beintegrated on the IC with instructions for operating or booting theimage processor 912. This ROM may be minimized to match a minimum sizeneeded to provide basic functionality for gathering sensor data fromvisible light camera 914 and infrared camera 920, such that no extrafunctionality that would cause delays in boot time are present. The ROMmay be configured with direct memory access (DMA) to the volatile memoryof the microprocessor of image processor 912. DMA allowsmemory-to-memory transfer of data from the ROM to system memory of theimage processor 912 independent of operation of a main controller ofimage processor 912. Providing DMA to this boot ROM further reduces theamount of time from power on of the image processor 912 until sensordata from the visible light camera 914 and infrared camera 920 can beprocessed and stored. In certain embodiments, minimal processing of thecamera signal from the visible light camera 914 and infrared camera 920is performed by the image processor 912, and additional processing maybe performed by applications operating on the mobile device 990 orserver system 998.

Low-power circuitry 920 includes low-power processor 922 and low-powerwireless circuitry 924. These elements of low-power circuitry 920 may beimplemented as separate elements or may be implemented on a single IC aspart of a system on a single chip. Low-power processor 922 includeslogic for managing the other elements of the eyewear device 910. Asdescribed above, for example, low power processor 922 may accept userinput signals from an interface 916. Low-power processor 922 may also beconfigured to receive input signals or instruction communications frommobile device 990 via low-power wireless connection 925. Additionaldetails related to such instructions are described further below.Low-power wireless circuitry 924 includes circuit elements forimplementing a low-power wireless communication system via a short-rangenetwork. Bluetooth™ Smart, also known as Bluetooth™ low energy, is onestandard implementation of a low power wireless communication systemthat may be used to implement low-power wireless circuitry 924. In otherembodiments, other low power communication systems may be used.

High-speed circuitry 930 includes high-speed processor 932, memory 934,and high-speed wireless circuitry 936. In the example, the infraredemitter 915 is shown as being coupled to the high-speed circuitry 930and operated by the high-speed processor 932. However, it should beunderstood that in some examples the infrared emitter 915 can be coupledto the low-power circuitry 920 such that the infrared emitter 915 isoperated by low-power processor 922. For example, a low-energy infraredbeam pattern can be emitted by the infrared emitter 915 with relativelyfew pixels in the matrix which equals less power and can also allow fora small package that fits into the design of the eyewear device 910,including the frame and chunks.

High-speed processor 932 may be any processor capable of managinghigh-speed communications and operation of any general computing systemneeded for eyewear device 910. High speed processor 932 includesprocessing resources needed for managing high-speed data transfers onhigh-speed wireless connection 937 to a wireless local area network(WLAN) using high-speed wireless circuitry 936. In certain embodiments,the high-speed processor 932 executes an operating system such as aLINUX operating system or other such operating system. In addition toany other responsibilities, the high-speed processor 932 executing asoftware architecture for the eyewear device 910 is used to manage datatransfers with high-speed wireless circuitry 936. In certainembodiments, high-speed wireless circuitry 936 is configured toimplement Institute of Electrical and Electronic Engineers (IEEE) 802.11communication standards, also referred to herein as Wi-Fi. In otherembodiments, other high-speed communications standards may beimplemented by high-speed wireless circuitry 936.

Memory 934 includes any storage device capable of storing camera datagenerated by the infrared camera 920, the visible light camera 914, andthe image processor 912. While memory 934 is shown as integrated withhigh-speed circuitry 930, in other embodiments, memory 934 may be anindependent standalone element of the eyewear device 910. In certainsuch embodiments, electrical routing lines may provide a connectionthrough a chip that includes the high-speed processor 932 from the imageprocessor 912 or low-power processor 922 to the memory 934. In otherembodiments, the high-speed processor 932 may manage addressing ofmemory 934 such that the low-power processor 922 will boot thehigh-speed processor 932 any time that a read or write operationinvolving memory 934 is needed.

FIG. 10 is a high-level functional block diagram of an example of amobile device 1090 that communicates via the user identification systemof FIG. 9. Shown are elements of a touch screen type of mobile device1090 having a user identification application 1045 loaded, althoughother non-touch type mobile devices can be used in the useridentification communications and controls under consideration here.Examples of touch screen type mobile devices that may be used include(but are not limited to) a smart phone, a personal digital assistant(PDA), a tablet computer, a laptop computer, or other portable device.However, the structure and operation of the touch screen type devices isprovided by way of example; and the subject technology as describedherein is not intended to be limited thereto. For purposes of thisdiscussion, FIG. 10 therefore provides a block diagram illustration ofthe example mobile device 1090 having a touch screen display fordisplaying content and receiving user input as (or as part of) the userinterface. Mobile device 1090 also includes a camera(s) 1070, such asvisible light camera(s).

The activities that are the focus of discussions here typically involvedata communications related to eye scanning for user identification andsecurity in a portable eyewear device. As shown in FIG. 10, the mobiledevice 1090 includes at least one digital transceiver (XCVR) 1010, shownas WWAN XCVRs, for digital wireless communications via a wide areawireless mobile communication network. The mobile device 1090 alsoincludes additional digital or analog transceivers, such as short rangeXCVRs 1020 for short-range network communication, such as via NFC, VLC,DECT, ZigBee, Bluetooth™, or WiFi. For example, short range XCVRs 1020may take the form of any available two-way wireless local area network(WLAN) transceiver of a type that is compatible with one or morestandard protocols of communication implemented in wireless local areanetworks, such as one of the Wi-Fi standards under IEEE 802.11 andWiMAX.

To generate location coordinates for positioning of the mobile device1090, the mobile device 1090 can include a global positioning system(GPS) receiver. Alternatively, or additionally the mobile device 1090can utilize either or both the short range XCVRs 1020 and WWAN XCVRs1010 for generating location coordinates for positioning. For example,cellular network, WiFi, or Bluetooth™ based positioning systems cangenerate very accurate location coordinates, particularly when used incombination. Such location coordinates can be transmitted to the eyeweardevice over one or more network connections via XCVRs 1020.

The transceivers 1010, 1020 (network communication interface) conformsto one or more of the various digital wireless communication standardsutilized by modern mobile networks. Examples of WWAN transceivers 1010include (but are not limited to) transceivers configured to operate inaccordance with Code Division Multiple Access (CDMA) and 3rd GenerationPartnership Project (3GPP) network technologies including, for exampleand without limitation, 3GPP type 2 (or 3GPP2) and LTE, at timesreferred to as “4G.” For example, the transceivers 1010, 1020 providetwo-way wireless communication of information including digitized audiosignals, still image and video signals, web page information for displayas well as web related inputs, and various types of mobile messagecommunications to/from the mobile device 1090 for user identificationstrategies.

Several of these types of communications through the transceivers 1010,1020 and a network, as discussed previously, relate to protocols andprocedures in support of communications with the eyewear device or theserver system for user identity verification utilizing eye scanners,e.g., infrared emitters and infrared cameras, to digitize and processimages of the retina or iris of the eye. Such communications, forexample, may transport packet data via the short range XCVRs 1020 overthe wireless connections 925 and 937 to and from the eyewear device asshown in FIG. 9. Such communications, for example, may also transportdata utilizing IP packet data transport via the WWAN XCVRs 1010 over thenetwork (e.g., Internet) 995 shown in FIG. 9. Both WWAN XCVRs 1010 andshort range XCVRs 1020 connect through radio frequency (RF)send-and-receive amplifiers (not shown) to an associated antenna (notshown).

The mobile device 1090 further includes a microprocessor, shown as CPU1030, sometimes referred to herein as the host controller. A processoris a circuit having elements structured and arranged to perform one ormore processing functions, typically various data processing functions.Although discrete logic components could be used, the examples utilizecomponents forming a programmable CPU. A microprocessor for exampleincludes one or more integrated circuit (IC) chips incorporating theelectronic elements to perform the functions of the CPU. The processor1030, for example, may be based on any known or available microprocessorarchitecture, such as a Reduced Instruction Set Computing (RISC) usingan ARM architecture, as commonly used today in mobile devices and otherportable electronic devices. Of course, other processor circuitry may beused to form the CPU 1030 or processor hardware in smartphone, laptopcomputer, and tablet.

The microprocessor 1030 serves as a programmable host controller for themobile device 1090 by configuring the mobile device to perform variousoperations, for example, in accordance with instructions or programmingexecutable by processor 1030. For example, such operations may includevarious general operations of the mobile device, as well as operationsrelated to user identification and communications with the eyeweardevice and server system. Although a processor may be configured by useof hardwired logic, typical processors in mobile devices are generalprocessing circuits configured by execution of programming.

The mobile device 1090 includes a memory or storage device system, forstoring data and programming. In the example, the memory system mayinclude a flash memory 1040A and a random access memory (RAM) 1040B. TheRAM 1040B serves as short term storage for instructions and data beinghandled by the processor 1030, e.g. as a working data processing memory.The flash memory 1040A typically provides longer term storage.

Hence, in the example of mobile device 1090, the flash memory 1040A isused to store programming or instructions for execution by the processor1030. Depending on the type of device, the mobile device 1090 stores andruns a mobile operating system through which specific applications,including user identification application 1045. Applications, such asthe user identification application 1045, may be a native application, ahybrid application, or a web application (e.g., a dynamic web pageexecuted by a web browser) that runs on mobile device 1090 to uniquelyidentify the user. Examples of mobile operating systems include GoogleAndroid, Apple iOS (I-Phone or iPad devices), Windows Mobile, AmazonFire OS, RIM BlackBerry operating system, or the like.

As shown, flash memory 1040A storage device stores a database ofcaptured infrared images of respective eyes of multiple users 1050. Thedatabase of captured infrared images of respective eyes of multipleusers 1050 is accumulated over time as different users of the eyeweardevice set up a profile in the user identification system. Initially,each user utilizes the eye scanner 113 to capture various images of aneye. The captured images are then populated into the database ofcaptured infrared images of respective eyes of multiple users 1050 toallow for user identification.

In the example, an eyewear device 100 captures a digital image ofreflection variations of the emitted pattern of infrared light 1060 andthe captured digital image in the flash memory 1040A. To uniquelyidentify the user of the eyewear device 100, current reflectionvariations of an emitted pattern of infrared light 1060 is compared bythe processor 1030 to previously captured infrared images of respectiveeyes of multiple users 1050 within the database to uniquely identify theuser of the eyewear device. It will be understood that the mobile device1090 is just one type of host computer in the user identification systemand that other arrangement may be utilized. For example, a server systemsuch as that shown in FIG. 9 may host the database of captured infraredimages of respective eyes of multiple users 1050 and perform thecomparison to make the unique user identification determination. Wherethe database of captured infrared images of respective eyes of multipleusers 1050 and the reflection variations of the emitted pattern ofinfrared light 1060 is stored and processed can vary depending on thesecurity preferences of the user and the system requirements.

The user identification application 1045 includes programming functionsto populate the database with captured infrared images of respectiveeyes of multiple users 1050 and to uniquely identify the user. Forexample, the programming functions may include comparing the digitalimage of reflection variations of the emitted pattern of infrared light1060 with the database of captured infrared images of respective eyes ofmultiple users 1050. In addition, any of the user identificationfunctionality described herein for the eyewear device, mobile device,and server system can be embodied in one more applications as describedpreviously.

FIG. 11A shows various alternate locations for the eye scanner on theeyewear device, which can be used individually or in combination. Ashown, multiple eyewear scanners 1113A-D can be included in the eyeweardevice 1100 to reduce errors in the user identification determinationand to determine a direction in which the user is looking (e.g., line ofsight) for eye tracking. In the example, there a four eye scanners1113A-D, and each eye scanner 1113A-D includes a respective infraredemitter 1115A-D and infrared camera 1120A-D.

As shown, the frame 1105 includes opposing first and second lateralsides 1170A-B. A first chunk 1110A is integrated into the first lateralside 1170A of frame 1105. A second chunk 1110B is integrated into thesecond lateral side 1170B of frame 1105. A circuit board (not shown)spans the first chunk 1110A, the frame 1105, and the second chunk 1110B.The frame 1105 of the eyewear device 1100 includes an upper frameportion 1195, a middle frame portion 1196, and a lower frame portion1197.

As depicted in FIG. 11A, eye scanner 1113A is located on the first rim1107A on the upper frame portion 1195. Eye scanner 1113B is located onthe second chunk 1110B. Eye scanner 1113C is located on the first rim1107A on the lower frame portion 1197. Eye scanner 1113D is located onthe first rim on the middle frame portion 1196.

Eyewear device 1100 includes a first eye scanner 1113A that includes afirst infrared emitter 1115A and a first infrared camera 1120A. Eyeweardevice 1100 also includes a second eye scanner 1113B that includes asecond infrared emitter 1115B and a second infrared camera 1120B. Thesecond infrared emitter 1115B is connected to the frame 1105 or the atleast one chunk 1110A-B to emit a second emitted pattern of infraredlight. The second infrared camera 1120B is connected to the frame 1105or the at least one chunk 1110A-B to capture reflection variations inthe second emitted pattern of infrared light. It should be understoodthat the first and second eye scanners can include any combination oflocations, or number of eye scanners 1113A-D shown in FIG. 11A,including one, two, three, or four of the eye scanners 1113A-D.Additionally, the eye scanners 1113A-D can be located on other portionsof the eyewear device 1100, including the first chunk 1110A; upper,middle, and lower portions 1195-1197 of the second rim 1107B; the bridge1106, or the temples.

Execution of the programming by a processor of a user identificationsystem, for example in the eyewear device 1100 or a coupled mobiledevice or server system, configures the system to perform functions. Inan example, the eyewear device 1110 emits, via the second infraredemitter 1115B, the second emitted pattern of infrared light on a secondeye of the user of the eyewear device 1110; captures, via the secondinfrared camera 1120B, reflection variations in the second emittedpattern of infrared light on a second eye of the user. Based on thereflection variations of the second emitted pattern of infrared light onthe second eye of the user, the system determines a direction of a lineof sight of the eyes of the user for eye tracking.

In another example, the eyewear device 1100 emits, via the secondinfrared emitter 1115B, the second emitted pattern of infrared light ona different portion of the eye of the user of the eyewear device 1100than the first infrared emitter 1115A. The eyewear device 1100 captures,via the second infrared camera 1115B, the reflection variations in thesecond emitted pattern of infrared light on the different portion of theeye of the user. Based on the reflection variations of the secondemitted pattern of infrared light on the different portion of the eye ofthe user, the system uniquely identifies the user of the eyewear device1100 based on the reflection variations of the second emitted pattern ofinfrared light on the different portion of the eye of the user. Thesecond emitted pattern of infrared light can be the same or differentfrom the first pattern of infrared light emitted by the first infraredemitter 1115A. The second infrared emitter 1115B and the second infraredcamera 1120B can be co-located on the frame 1105 or the at least onechunk 1110A-B as shown in FIG. 11. Although not shown in FIG. 11A, thefirst infrared emitter 1115A and the infrared camera 1120A can beco-located on a first chunk 1110A. The second infrared emitter 1115B andthe second infrared camera 1120B can be co-located on a second chunk1110B.

As described and depicted in FIG. 1 and shown in FIG. 11A, the frame1105 of the eyewear device 1100 includes first and second eye rims1107A-B that have respective apertures to hold a respective opticalelement and the first and second eye rims 1107A-B are connected by abridge 1106. In an example, the first infrared emitter 1115A and thefirst infrared camera 1120A are co-located on the first eye rim 1107A.Although not shown in FIG. 11B, the second infrared emitter 1115B andthe second infrared camera 1120B can be co-located on the second eye rim1107B, including on the upper frame portion 1195, middle frame portion1196, and lower frame portion 1197.

FIGS. 11B-D illustrate the effects of the various alternate locations onthe eyewear device with respect to different orientations of the eye ofthe user. In FIG. 11B, the eye of the user 1180B is looking up.Accordingly, placement of the eye scanner 1113A, such as the infraredemitter and infrared camera, on either the upper frame portion (e.g.,top frame on the rims, bridge, etc.) or a chunk can accurately capturean image of the retina or iris of the eye of the user 1180B looking up.Also, placement of the eye scanner 1113B on a lower frame portion (e.g.,bottom frame) of the eyewear device also accurately captures an image ofthe retina or iris of the eye of the user 1180B looking up. Hence bothfields of view are depicted as suitable (OK).

In FIG. 11C, the eye of the user 1180C is looking straight ahead. Inthis scenario, again placement of the eye scanner 1113A on either theupper frame portion or a chunk can accurately capture an image of theretina or iris of the eye of the user 1180C looking straight ahead.Also, placement of the eye scanner 1113B on the lower frame portion ofthe eyewear device accurately captures an image of the retina or iris ofthe eye of the user 1180C looking straight ahead.

In FIG. 11D, the eye of the user 1180D is looking down. In thisorientation of the eye of the user 1180D, placement of the eye scanner1113A, on either the upper frame portion or a chunk may be insufficientbecause the eyelid of the user 1180D can block the infrared camera.Hence the field of view is depicted as not good (NG). However, placementof the eye scanner 1113B on the lower frame portion of the eyeweardevice can accurately capture an image of the retina or iris of the eyeof the user 1180D looking down. Thus, having multiple eye scanners1113A-B on the eyewear device can improve performance of the useridentification system by improving accuracy and reducing errors in eyescanning. In addition, multiple eye scanners 1113A-B can be used for eyetracking directional information, for example, to detect where the useris looking (left, right, up, down, east, west, north, south, etc.).

In an example, location coordinates of the user of the eyewear devicecan also be generated by the location sensor components of the eyeweardevice or a mobile device being carried by the user that is incommunication via the connections 925 and 937 as described in FIGS.9-10. With the eye tracking directional information along with thelocation coordinates of the user, specific content can be delivered tothe eyewear device. For example, if the user is walking down the streetand looking at a store, the eyewear device can be loaded withinformation about the particular store for delivering coupons formonetizing purposes.

According to some embodiments, an “application” or “applications” areprogram(s) that execute functions defined in the programs. Variousprogramming languages can be employed to create one or more of theapplications, structured in a variety of manners, such asobject-oriented programming languages (e.g., Objective-C, Java, or C++)or procedural programming languages (e.g., C or assembly language). In aspecific example, a third party application (e.g., an applicationdeveloped using the ANDROID™ or IOS™ software development kit (SDK) byan entity other than the vendor of the particular platform) may bemobile software running on a mobile operating system such as IOS™,ANDROID™ WINDOWS® Phone, or another mobile operating systems. In thisexample, the third party application can invoke API calls provided bythe operating system to facilitate functionality described herein.

FIG. 12 is a flowchart of the operation of the eyewear device and othercomponents of the user identification system. As noted above, utilizingthe eyewear devices and protocols and procedures of the useridentification system described herein, the identity of a user can beverified. Although shown as occurring serially, the blocks of FIG. 12may be reordered or parallelized depending on the implementation.

Beginning in block 1200, eyewear device initiates scanning of the eye.In one example, eye scanning is initiated when a user puts the eyeweardevice on, for example, over the user's eyes. Such wearing of theeyewear device can be detected after the eyewear device detects that thetemples have been be unfolded via an open/close sensor (e.g., magneticcontacts) mounted on a circuit board that is coupled to the temples andhinges. Or, for example, a capacitive strip on the bridge, temples, orother portions of the eyewear device may detect that the eyewear deviceis being worn by the user. In response to detecting wearing of theeyewear device, for example, for a predetermined time, the remainingblocks of FIG. 12 may be executed. In another example, eye scanning isinitiated within a predetermined time period after the eyewear device ispowered on. In another example, eye scanning is initiated when anotherfunction of the eyewear device is triggered, for example, a differentsoftware executable application is accessed which requires appropriateuser or group permissions.

Eye scanning can be initiated when hardware is accessed on the eyeweardevice, for example, when a button is pressed to capture images or avideo via the visible light camera or another user interface orcomponent of the eyewear device is utilized. In another embodiment, theeyewear device initiates an eye scan under certain conditions (e.g.,detection of motion from an on-board accelerometer or gyroscope) ordetecting modification of positional location coordinates via a GPSreceiver or other positioning system.

Continuing to block 1210, the eye scanner of the eyewear device emits apattern of infrared light. As described in detail previously, theinfrared emitter emits the pattern of infrared light which can be astandardized matrix or beam of pixels that will outline a uniform lighttrace on the eye of the user (e.g., retina or iris). The emitted patterncan be an unperceived low-energy infrared beam that shines on the eyewith a standardized path.

Proceeding to block 1220, the eyewear device captures reflectionvariations in the emitted pattern of infrared light. As outlined above,the amount of reflection of the emitted pattern of infrared light variesin different parts of the retina (e.g., retinal blood vessels absorblight more than surrounding tissue) and the iris. The infrared cameracaptures these reflection variations of the emitted pattern of infraredlight, which is digitized by the eyewear device.

Moving to block 1230, a user of the eyewear device is identified basedon the currently captured digitized reflection variations, on one ormore devices of the user identification system, such as the eyeweardevice, mobile device, or server system. A database with the digitizedreflection variations that were previously stored are analyzed usingalgorithms to compare against the currently captured digitizedreflection variations. The algorithms employ mathematical andstatistical techniques for pattern recognition to determine whether thecurrently captured reflection variations of the user of the eyeweardevice matches one or more of the previously captured digitized imagesthat are stored and exist in the database. If a match is found, theidentity of the user is verified (e.g., known) and corresponding useraccount information is retrieved.

Finishing now in block 1240, actions are taken based on identificationor lack of identification of the user. For example, the eyewear deviceand associated mobile device may be unlocked and profile settings orconfigurations of the eyewear device can be loaded based on theassociated user account. In one example, access to certain softwareexecutable applications and associated hardware can be granted, such asthe visible light camera, of the eyewear device. In another example, theeyewear device may automatically pair with the mobile device associatedwith the identified user account in response to user identification. Insome embodiments, the user may be automatically be logged into useraccounts on third party software applications, for example, anapplication store or chat application. In other examples, the identityof the user or the identity of the user account can be included in themetadata of images or videos captured by the visible light camera alongwith geolocation data.

If the user is not identified (e.g., no match is found in the database),then the eyewear device and mobile device may remain locked andinaccessible. For example, the eyewear device and mobile device lockdown and the account associated with the devices receives messages thatthere was a non-matching access to the devices. Alternatively, if thisis the first time the user utilizes the user identification system, thesystem will find no previously captured infrared image exists in thedatabase with digitized reflection variations that match the currentlycaptured reflection variations of the emitted pattern of infrared light.In response to finding no matching captured infrared image exists, thesystem may update the database to store digitized images of thecurrently captured reflection variations of the emitted pattern ofinfrared light. The system may then allow the user access to the eyeweardevice and mobile device, for example, and request that the user set upa user account.

It will be understood that the terms and expressions used herein havethe ordinary meaning as is accorded to such terms and expressions withrespect to their corresponding respective areas of inquiry and studyexcept where specific meanings have otherwise been set forth herein.Relational terms such as first and second and the like may be usedsolely to distinguish one entity or action from another withoutnecessarily requiring or implying any actual such relationship or orderbetween such entities or actions. The terms “comprises,” “comprising,”“includes,” “including,” or any other variation thereof, are intended tocover a non-exclusive inclusion, such that a process, method, article,or apparatus that comprises or includes a list of elements or steps doesnot include only those elements or steps but may include other elementsor steps not expressly listed or inherent to such process, method,article, or apparatus. An element preceded by “a” or “an” does not,without further constraints, preclude the existence of additionalidentical elements in the process, method, article, or apparatus thatcomprises the element.

Unless otherwise stated, any and all measurements, values, ratings,positions, magnitudes, sizes, and other specifications that are setforth in this specification, including in the claims that follow, areapproximate, not exact. Such amounts are intended to have a reasonablerange that is consistent with the functions to which they relate andwith what is customary in the art to which they pertain. For example,unless expressly stated otherwise, a parameter value or the like mayvary by as much as ±10% from the stated amount.

In addition, in the foregoing Detailed Description, it can be seen thatvarious features are grouped together in various examples for thepurpose of streamlining the disclosure. This method of disclosure is notto be interpreted as reflecting an intention that the claimed examplesrequire more features than are expressly recited in each claim. Rather,as the following claims reflect, the subject matter to be protected liesin less than all features of any single disclosed example. Thus, thefollowing claims are hereby incorporated into the Detailed Description,with each claim standing on its own as a separately claimed subjectmatter.

While the foregoing has described what are considered to be the bestmode and other examples, it is understood that various modifications maybe made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that they may be appliedin numerous applications, only some of which have been described herein.It is intended by the following claims to claim any and allmodifications and variations that fall within the true scope of thepresent concepts.

What is claimed is:
 1. A system comprising: an eyewear device including:a frame; a first temple connected to a first lateral side of the frame;a second temple connected to a second lateral side of the frame; a firstinfrared emitter connected to the frame or the first temple to emit afirst pattern of infrared light; a second infrared emitter connected tothe frame, the first temple, or the second temple to emit a secondpattern of infrared light; a first infrared camera connected to theframe or the first temple to capture reflection variations in the firstemitted pattern of infrared light; and a second infrared cameraconnected to the frame, the first temple, or the second temple tocapture reflection variations in the second emitted pattern of infraredlight; a processor coupled to the eyewear device; a memory accessible tothe processor; and programming in the memory, wherein execution of theprogramming by the processor configures the system to perform functions,including functions to: emit, via the first infrared emitter, the firstpattern of infrared light on a first eye of a user of the eyeweardevice; capture, via the first infrared camera, the reflectionvariations in the first emitted pattern of infrared light on the firsteye of the user; emit, via the second infrared emitter, the secondpattern of infrared light on a second eye of the user of the eyeweardevice; capture, via the second infrared camera, the reflectionvariations in the second emitted pattern of infrared light on the secondeye of the user; and determine a direction of a line of sight of thefirst and second eyes of the user for eye tracking based on at least thereflection variations in the second emitted pattern of infrared light onthe second eye of the user.
 2. The system of claim 1, wherein executionof the programming by the processor configures the system to performadditional functions, including functions to: receive locationcoordinates of the user of the eyewear device; and provide content tothe eyewear device based on the location coordinates and the directionof the line of sight of the first and second eyes of the user.
 3. Thesystem of claim 1, wherein the first infrared emitter and first infraredcamera and are co-located on the frame.
 4. The system of claim 3,wherein the second infrared emitter and the second infrared camera areco-located on the frame, the first temple, the second temple, or a chunkthat is integrated into or connected to the frame on the first or secondlateral side.
 5. The system of claim 4, wherein when the first infraredcamera cannot capture an image of the retina or iris of the eye of theuser due to the orientation of the eye of the user, the second infraredcamera is positioned to capture the image of the retina of the iris ofthe eye of the user.
 6. The system of claim 1, wherein: the firstinfrared emitter and the first infrared camera are co-located on a firstchunk that is integrated into or connected to the frame on the firstlateral side.
 7. The system of claim 6, wherein: the second infraredemitter and the second infrared camera are co-located on a second chunkthat is integrated into or connected to the frame on the second lateralside.
 8. The system of claim 6, wherein the first chunk includes acircuit board that includes the first infrared emitter and the firstinfrared camera.
 9. The system of claim 8, wherein the frame includes: aframe front; and a frame back; and the circuit board includes isflexible printed circuit board disposed between the frame front and theframe back.
 10. The system of claim 9, wherein: the first infraredcamera is disposed on the circuit board and is covered by an infraredcamera cover lens; and the frame back includes a first opening for theinfrared camera cover lens.
 11. The system of claim 1, wherein: theframe includes first and second eye rims that have respective aperturesto hold a respective optical element; the first and second eye rims areconnected by a bridge; the first infrared emitter and the first infraredcamera are co-located on the first eye rim; and the second infraredemitter and the second infrared camera are co-located on the second eyerim.
 12. The system of claim 1, further comprising: a plurality ofinfrared emitters connected to the frame, the first temple, the secondtemple, a first chunk that is integrated into or connected to the frameon the first lateral side, or a second chunk that is integrated orconnected to the frame on the second lateral side, the plurality ofinfrared emitters including the first and second infrared emitters; anda plurality of infrared cameras connected to the frame, the firsttemple, the second temple, the first chunk, or the second chunk, theplurality of infrared cameras including the first and second infraredcameras, wherein the plurality of infrared emitters and plurality ofinfrared cameras track eye direction of the user to detect whether theuser is looking left, right, up, down, east, west, north, or south. 13.A method for tracking the eyes of a user of an eyewear device, theeyewear device including a first infrared emitter, a second infraredemitter, a first infrared camera, and a second infrared camera, themethod comprising: emitting, via the first infrared emitter, a firstpattern of infrared light on a first eye of a user of the eyeweardevice; capturing, via the first infrared camera, reflection variationsin the first emitted pattern of infrared light on the first eye of theuser; emitting, via the second infrared emitter, a second pattern ofinfrared light on a second eye of the user of the eyewear device;capturing, via the second infrared camera, reflection variations in thesecond emitted pattern of infrared light on the second eye of the user;and determining a direction of a line of sight of the first and secondeyes of the user for eye tracking based on at least the reflectionvariations in the second emitted pattern of infrared light on the secondeye of the user.
 14. The method of claim 13, further comprising:receiving location coordinates of the user of the eyewear device; andproviding content to the eyewear device based on the locationcoordinates and the direction of the line of sight of the first andsecond eyes of the user.
 15. The method of claim 13, wherein the firstinfrared emitter and first infrared camera and are co-located on a frameof the eyewear device and the second infrared emitter and the secondinfrared camera are co-located on the frame, a temple, or a chunk thatis integrated into or connected to the frame on a lateral side of theeyewear device, further comprising: capturing an image of the retina oriris of the eye of the user using the second infrared emitter and secondinfrared camera when the first infrared camera and first infraredemitter cannot capture the image of the retina or iris of the eye of theuser due to the orientation of the eye of the user.
 16. The method ofclaim 13, wherein a plurality of infrared emitters are connected to theframe, the first temple, the second temple, a first chunk that isintegrated into or connected to the frame on the first lateral side, ora second chunk that is integrated or connected to the frame on thesecond lateral side, the plurality of infrared emitters including thefirst and second infrared emitters, and wherein a plurality of infraredcameras are connected to the frame, the first temple, the second temple,the first chunk, or the second chunk, the plurality of infrared camerasincluding the first and second infrared cameras, further comprising:tracking, using the plurality of infrared emitters and plurality ofinfrared cameras, eye direction of the user to detect whether the useris looking left, right, up, down, east, west, north, or south.
 17. Anon-transitory computer readable medium including instructions forimplementing functions when executed by a processor of an eyewear deviceincluding a first infrared emitter, a second infrared emitter, a firstinfrared camera, and a second infrared camera, the functions comprising:emitting, via the first infrared emitter, a first pattern of infraredlight on a first eye of a user of the eyewear device; capturing, via thefirst infrared camera, reflection variations in the first emittedpattern of infrared light on the first eye of the user; emitting, viathe second infrared emitter, a second pattern of infrared light on asecond eye of the user of the eyewear device; capturing, via the secondinfrared camera, reflection variations in the second emitted pattern ofinfrared light on the second eye of the user; and determining adirection of a line of sight of the first and second eyes of the userfor eye tracking based on at least the reflection variations in thesecond emitted pattern of infrared light on the second eye of the user.18. The computer readable medium of claim 17, further comprisinginstructions for implementing functions when executed by the processorof the eyewear device, the functions including: receiving locationcoordinates of the user of the eyewear device; and providing content tothe eyewear device based on the location coordinates and the directionof the line of sight of the first and second eyes of the user.
 19. Thecomputer readable medium of claim 17, wherein the first infrared emitterand first infrared camera and are co-located on a frame of the eyeweardevice and the second infrared emitter and the second infrared cameraare co-located on the frame, a temple, or a chunk that is integratedinto or connected to the frame on a lateral side of the eyewear device,further comprising instructions for implementing functions when executedby the processor of the eyewear device, the functions including:capturing an image of the retina or iris of the eye of the user usingthe second infrared emitter and second infrared camera when the firstinfrared camera and first infrared emitter cannot capture the image ofthe retina or iris of the eye of the user due to the orientation of theeye of the user.
 20. The computer readable medium of claim 17, wherein aplurality of infrared emitters are connected to the frame, the firsttemple, the second temple, a first chunk that is integrated into orconnected to the frame on the first lateral side, or a second chunk thatis integrated or connected to the frame on the second lateral side, theplurality of infrared emitters including the first and second infraredemitters, further comprising instructions for implementing functionswhen executed by the processor of the eyewear device, the functionsincluding: tracking, using the plurality of infrared emitters andplurality of infrared cameras, eye direction of the user to detectwhether the user is looking left, right, up, down, east, west, north, orsouth.